Anatomical visualization system

ABSTRACT

An improved anatomical visualization system which is adapted to augment a standard video endoscopic system with a coordinated computer model visualization system so as to enhance a physician&#39;s understanding of the patient&#39;s interior anatomical structure.

FIELD OF THE INVENTION

[0001] This invention relates to medical apparatus in general, and moreparticularly to anatomical visualization systems.

BACKGROUND OF THE INVENTION

[0002] Endoscopic surgical procedures are now becoming increasinglypopular due to the greatly reduced patient recovery times resulting fromsuch surgery.

[0003] More particularly, in endoscopic surgical procedures, relativelynarrow surgical instruments are inserted into the interior of thepatient's body so that the distal (i.e., working) ends of theinstruments are positioned at a remote interior surgical site, while theproximal (i.e., handle) ends of the instruments remain outside thepatient's body. The physician then manipulates the proximal (i.e.,handle) ends of the instruments as required so as to cause the distal(i.e., working) ends of the instruments to carry out the desiredsurgical procedure at the remote interior surgical site. As a result ofthis technique, the incisions made in the patient's body can remainrelatively small, thereby resulting in significantly faster patientrecovery times.

[0004] By way of example, laparoscopic surgical procedures have beendeveloped wherein the abdominal region of.the patient is inflated withgas (e.g., CO₂) and then surgical instruments are inserted into theinterior of the abdominal cavity so as to carry out the desired surgicalprocedure. By way of further example, arthroscopic surgical procedureshave been developed wherein a knee joint is inflated with a fluid (e.g.,a saline solution) and then surgical instruments are inserted into theinterior of the joint so as to carry out the desired surgical procedure.

[0005] In order to visualize what is taking place at the remote interiorsite, the physician also inserts an endoscope into the patient's bodyduring the endoscopic surgery, together with an appropriate source ofillumination. Such an endoscope generally comprises an elongated shafthaving a distal end and a proximal end, and at least one internalpassageway extending between the distal end and the proximal end. Imagecapturing means are disposed at the distal end of the shaft and extendthrough the shaft's at least one internal passageway, whereby the imagecapturing means can capture an image of a selected region locatedsubstantially adjacent to the distal end of the shaft and convey thatimage to the proximal end of the shaft. Viewing means are in turndisposed adjacent to the proximal end of the shaft, whereby the imageobtained by the image capturing means can be conveyed to a displaydevice which is viewed by the physician.

[0006] Endoscopes of the sort described above are generally sufficientto permit the physician to carry out the desired endoscopic procedure.However, certain problems have been encountered when using suchendoscopes in surgical procedures.

[0007] For example, endoscopes of the sort described above generallyhave a fairly limited field of view. As a result, the physiciantypically cannot view the entire surgical field in a single image. Thiscan mean that the physician may not see an important development as soonas it occurs, and/or that the physician must expend precious time andenergy constantly redirecting the endoscope to different anatomicalregions.

[0008] Visualization problems can also occur due to the difficulty ofproviding proper illumination within a remote interior site.

[0009] Also, visualization problems can occur due to the presence ofintervening structures (e.g., fixed anatomical structures, movingdebris, flowing blood, the presence of vaporized tissue when cauterizingin laparoscopic surgery, the presence of air bubbles in a liquid mediumin the case of arthroscopic surgery, etc.).

[0010] It has also been found that it can be very difficult for thephysician to navigate the endoscope about the anatomical structures ofinterest, due to the relative ambiguity of various anatomical structureswhen seen through the endoscope's aforementioned limited field of viewand due to the aforementioned visualization problems.

OBJECTS OF THE INVENTION

[0011] Accordingly, one object of the present invention is to provide animproved anatomical visualization system.

[0012] Another object of the present invention is to provide an improvedanatomical visualization system which is adapted to enhance aphysician's ability to comprehend the nature and location of internalbodily structures during endoscopic visualization.

[0013] Still another object of the present invention is to provide animproved anatomical visualization system which is adapted to enhance aphysician's ability to navigate an endoscope within the body.

[0014] Yet another object of the present invention is to provide animproved anatomical visualization system which is adapted to augment astandard video endoscopic system with a coordinated computer modelvisualization system so as to enhance the physician's understanding ofthe patient's interior anatomical structures.

[0015] And another object of the present invention is to provide animproved method for visualizing the interior anatomical structures of apatient.

[0016] And still another object of the present invention is to providean improved anatomical visualization system which can be used withremote visualization devices other than endoscopes, e.g., miniatureultrasound probes.

[0017] And yet another object of the present invention is to provide animproved visualization system which can be used to visualize remoteobjects other than interior anatomical structures, e.g., the interiorsof complex machines.

[0018] And another object of the present invention is to provide animproved method for visualizing objects.

SUMMARY OF THE INVENTION

[0019] These and other objects of the present invention are addressed bythe provision and use of an improved anatomical visualization systemcomprising, in one preferred embodiment, a database of pre-existingsoftware objects, wherein at least one of the software objectscorresponds to a physical structure which is to be viewed by the system;a real-time sensor for acquiring data about the physical structure whenthe physical structure is located within that sensor's data acquisitionfield, wherein the real-time sensor is capable of being moved aboutrelative to the physical structure; generating means for generating areal-time software object corresponding to the physical structure, usingdata acquired by the sensor; registration means for positioning thereal-time software object in registration with the pre-existing softwareobjects contained in the database; and processing means for generatingan image from the software objects contained in the database, based upona specified point of view.

[0020] In another preferred form of the invention, the generating meanscreate a software object that corresponds to a disk. The generatingmeans may also be adapted to texture map the data acquired by the sensoronto the disk. Also, the registration means may comprise tracking meansthat are adapted so as to determine the spatial positioning andorientation of the real-time sensor and/or the physical structure.

[0021] In another preferred aspect of the invention, the real-timesensor may comprise an endoscope and the physical structure may comprisean interior anatomical structure. The system may also include eitheruser input means for permitting the user to provide the processing meanswith the specified point of view, or user tracking means that areadapted to provide the processing means with the specified point ofview.

[0022] According to another aspect of the invention, the real-timecomputer-based viewing system may comprise a database of softwareobjects and image generating means for generating an image from thesoftware objects contained in the database, based upon a specified pointof view. In accordance with this aspect of the invention, means are alsoprovided for specifying this point of view. At least one of the softwareobjects contained in the database comprises pre-existing datacorresponding to a physical structure which is to be viewed by thesystem, and at least one of the software objects comprises datagenerated by a real-time, movable sensor. The system further comprisesregistration means for positioning the at least one software object,comprising data generated by the real-time movable sensor, inregistration with the at least one software object comprisingpre-existing data corresponding to the physical structure which is to beviewed by the system.

[0023] A preferred method for utilizing the present inventioncomprises:(1) positioning the sensor so that the physical structure islocated within that sensor's data acquisition field, and generating areal-time software object corresponding to the physical structure usingdata,acquired by the sensor, and positioning the real-time softwareobject in registration with the pre-existing software objects containedin the database; (2) providing a specified point of view to theprocessing means; and (3) generating an image from the software objectscontained in the database according to that specified point of view.

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] These and other objects and features of the present inventionwill be more fully disclosed or rendered obvious by the followingdetailed description of the preferred embodiment of the invention, whichis to be considered together with the accompanying drawings wherein:

[0025]FIG. 1 is a schematic view showing an anatomical visualizationsystem formed in accordance with the present invention;

[0026]FIG. 2 is a schematic view of a unit cube for use in definingpologonal surface models;

[0027]FIG. 3 illustrates the data file format of the pologonal surfacemodel for the simple unit cube shown in FIG. 2;

[0028]FIG. 4 illustrates a system of software objects;

[0029]FIG. 5 illustrates an image rendered by the anatomicalvisualization system;

[0030]FIG. 6 illustrates how various elements of system data are inputinto computer means 60 in connection with the sytem's generation ofoutput video for display on video display 170;

[0031]FIG. 7 illustrates additional details on the methodology employedby anatomical visualization system 10 in connection in rendering a videooutput image for display on video display 170;

[0032]FIG. 8 illustrates a typical screen display provided in accordancewith the present invention;

[0033]FIG. 9 illustrates an image rendered by the anatomicalvisualization system;

[0034]FIG. 10 is a schematic representation of a unit disk softwareobject where the disk is defined in the X-Y plane and has a diameter of1; and

[0035]FIG. 11 shows how the optical parameters for an encdoscope candefine the relationship between the endoscope 90A′ and the disk 90B′.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0036] Looking first at FIG. 1, there is shown an anatomicalvisualization system 10 which comprises a preferred embodiment of thepresent invention. Anatomical visualization system 10 is intended to beused by a physician 20 to visually inspect anatomical objects 30 locatedat an interior anatomical site. By way of example, anatomicalvisualization system 10 might be used by physician 20 to visuallyinspect a tibia 30A, a femur 30B and a meniscus 30C located within theknee joint of a patient.

[0037] An important aspect of the present invention is the provision ofan improved anatomical visualization system which is adapted to augmenta standard video endoscopic system with a coordinated computer modelvisualization system so as to enhance the physician's understanding ofthe patient's interior anatomical structure.

[0038] To that end, anatomical visualization system 10 generallycomprises endoscope means 40, endoscope tracking means 50, computermeans 60, database means 70 containing 3-D computer models of variousobjects which are to be visualized by the system, and display means 80.

[0039] Endoscope means 40 comprise an endoscope of the sort well knownin the art. More particularly, endoscope means 40 comprise an endoscope90 which comprises (i) a lens arrangement which is disposed at thedistal end of the endoscope for capturing an image of a selected regionlocated substantially adjacent to the distal end of the endoscope, and(ii) an appropriate image sensor, e.g., a charge coupled device (“CCD”)element or video tube, which is positioned on the endoscope so as toreceive an image captured by the lens arrangement and to generatecorresponding video signals which are representative of the capturedimage.

[0040] The video signals output from endoscope 90 are fed as an inputinto computer means 60. However, inasmuch as endoscope 90 will generallyoutput its video signals in analog form and inasmuch as computer means60 will generally require its video signal input to be in digital form,some conversion of the endoscope's video feed is generally required. Inthe preferred embodiment, video processing means 95 are provided toconvert the analog video signals output by endoscope 90 into the digitalvideo signals required by computer means 60. Video processing means 95are of the sort well known in the art and hence need not be described infurther detail here.

[0041] Endoscope tracking means 50 comprise a tracking system of thesort well known in the art. More particularly, endoscope tracking means50 may comprise a tracking system 97 of the sort adapted to monitor theposition and orientation of an object in space and to generate outputsignals which are representative of the position and orientation of thatobject. By way of example, tracking system 97 might comprise an opticaltracking system, an electromagnetic tracking system, an ultrasonictracking system, or an articulated linkage tracking system, among otheralternatives. Such tracking systems are all well known in the art andhence need not be described in further detail here. Tracking system 97is attached to endoscope 90 such that the output signals generated bytracking system 97 will be representative of the spatial positioning andorientation of endoscope 90. The output signals generated by trackingsystem 97 is fed as an input into computer means 60.

[0042] Computer means 60 comprise a digital computer 130 of the sortadapted for high speed processing of computer graphics. Such digitalcomputers are well known in the art. By way of example, digital computer130 might comprise a Silicon Graphics Reality Engine digital computer,or it might comprise a Silicon Graphics Iris Indigo² Impact digitalcomputer, or it might comprise some equivalent digital computer.

[0043] Computer means 60 also comprise the operating system software(schematically represented at 135 in FIG. 1) and the application programsoftware (schematically represented at 140 in FIG. 1) required to causecomputer 130 to operate in the manner hereinafter described. Inparticular, application program software 140 includes image renderingsoftware of the sort adapted to generate images from the 3-D computermodels contained in database means 70 according to a specified point ofview. By way of example, where digital computer 130 comprises a SiliconGraphics digital computer of the sort disclosed above, operating systemsoftware 135 might comprise the IRIX operating system, and the imagerendering software contained in application program software 140 mightcomprise the IRIS gl image rendering software or the OpenGL imagerendering software. Such software is well know in the art. As is alsowell known in the art, such image rendering software utilizes standardtechniques, such as the well-known Z buffer algorithm, to draw images of3-D computer models according to some specified point of view.

[0044] As is well known in the art, computer 130 also typically includesinput devices 145 through which physician 20 can interact with thecomputer. Input devices 145 preferably comprise the sort of computerinput devices generally associated with a Silicon Graphics digitalcomputer, e.g., input devices 145 preferably comprise a keyboard, amouse, etc. Among other things, input devices 145 permit physician 20 toinitiate operation of anatomical visualization system 10, to selectvarious system functions, and to supply the system with variousdirectives, e.g., input devices 145 might be used by physician 20 tospecify a particular viewing position for which the applicationprogram's image rendering software should render a visualization of the3-D software models contained in database means 70.

[0045] Database means 70 comprise a data storage device or medium 150containing one or more 3-D computer models (schematically illustrated as160 in FIG. 1) of the anatomical objects 30 which are to be visualizedby anatomical visualization system 10. The specific data structure usedto store the 3-D computer models 160 will depend on the specific natureof computer 130 and on the particular operating system software 135 andthe particular application program software 140 being run on computer130. In general, however, the 3-D computer models 160 contained in datastorage device or medium 150 are preferably structured as a collectionof software objects. By way of example, a scanned anatomical structuresuch as a human knee might be modeled as three distinct softwareobjects, with the tibia being one software object (schematicallyrepresented at 30A′ in FIG. 4), the femur being a second software object(schematically represented at 30B′ in FIG. 4), and the meniscus being athird software object (schematically represented at 30C′ in FIG. 4).Such software objects are of the sort well known in the art and may havebeen created, for example, through post-processing of CT or MRI scans ofthe patient using techniques well known in the art.

[0046] By way of example, in the case where digital computer 130comprises a Silicon Graphics digital computer of the sort describedabove, and where the operating systems's software comprises the IRIXoperating system and the application program's image rendering softwarecomprises the Iris gl or OpenGL image rendering software, the 3-Dcomputer models 160 might comprise software objects defined as polygonalsurface models, since such a format is consistent with theaforementioned software. By way of further example, FIGS. 2 and 3illustrate a typical manner of defining a software object using apolygonal surface model of the sort utilized by such image renderingsoftware. In particular, FIG. 2 illustrates the vertices of a unit cubeset in an X-Y-Z coordinate system, and FIG. 3 illustrates the data fileformat of the polygonal surface model for this simple unit cube. As iswell known in the art, more complex shapes such as human anatomicalstructures can be expressed in corresponding terms. It is also to beappreciated that certain digital computers, such as a Silicon Graphicsdigital computer of the sort described above, can be adapted such thatdigital video data of the sort output by video processing means 95 canbe made to appear on the surface of a polygonal surface model softwareobject in the final rendered image using the well known technique oftexture mapping.

[0047] Display means 80 comprise a video display of the sort well knownin the art. More particularly, display means 80 comprise a video display170 of the sort adapted to receive video signals representative of animage and to display that image on a screen 180 for viewing by physician20. By way of example, video display 170 might comprise a televisiontype of monitor, or it might comprise a head-mounted display or aboom-mounted display, or it might comprise any other display device ofthe sort suitable for displaying an image corresponding to the videosignals received from computer means 60, as will hereinafter bedescribed in further detail. In addition, where video display 170comprises a head-mounted display or a boom-mounted display or some othersort of display coupled to the physician's head movements, physiciantracking means 185 (comprising a tracking system 187 similar to thetracking system 97 described above) may be attached to video display 170and then used to advise computer 130 of the physician's head movements.This can be quite useful, since the anatomical visualization system 10can use such physician head movements to specify a particular viewingposition for which the application program's image rendering softwareshould render a visualization of the 3-D software models contained indatabase means 70.

[0048] In addition to the foregoing, it should also be appreciated thatsurgeon tracking means 188 (comprising a tracking system 189 similar tothe tracking system 97 described above) may be attached directly tosurgeon 20 and then used to advise computer 130 of the physician'smovements. Again, the anatomical visualization system can use suchphysician movements to specify a particular viewing position for whichthe application program's image rendering software should render avisualization of the 3-D software models contained in database means 70.

[0049] As noted above, an important aspect of the present invention isthe provision of an improved anatomical visualization system which isadapted to augment a standard video endoscopic system with a coordinatedcomputer model visualization system so as to enhance the physician'sunderstanding of the patient's interior anatomical structure. Inparticular, the improved anatomical visualization system is adapted toaugment the direct, but somewhat limited, video images generated by astandard video endoscopic system with the indirect, but somewhat moreflexible, images generated by a computer model visualization system.

[0050] To this end, and referring now to FIG. 1, database means 70 alsocomprise one or more 3-D computer models (schematically illustrated at190 in FIG. 1) of the particular endoscope 90 which is included inanatomical visualization system 10. Again, the specific data structureused to store the 3-D computer models 190 representing endoscope 90 willdepend on the specific nature of computer 130 and on the particularoperating system software 135 and the particular application programsoftware 140 being run on computer 130. In general, however, the 3-Dcomputer models 190 contained in data storage device or medium 150 arepreferably structured as a pair of separate but interrelated softwareobjects, where one of the software objects represents the physicalembodiment of endoscope 90, and the other of the software objectsrepresents the video image acquired by endoscope 90.

[0051] More particularly, the 3-D computer models 190 representingendoscope 90 comprises a first software object (schematicallyrepresented at 90A′ in FIG. 4) representative of the shaft of endoscope90.

[0052] The 3-D computer models 190 representing endoscope 90 alsocomprises a second software object (schematically represented at 90B′ inFIG. 4) which is representative of the video image acquired by endoscope90. More particularly, second software object 90B′ is representative ofa planar disk defined by the intersection of the endoscope's field ofview with a plane set perpendicular to the center axis of that field ofview, wherein the plane is separated from the endoscope by a distanceequal to the endoscope's focal distance. See, for example, FIG. 11,which shows how the optical parameters for an endoscope can define therelationship between the endoscope 90A′ and the disk 90B′. In addition,and as will hereinafter be described in further detail, the anatomicalvisualization system 10 is arranged so that the video signals output byendoscope 90 are, after being properly transformed by video processingmeans 95 into the digital data format required by digital computer 130,texture mapped onto the planar surface of disk 90B′. Thus it will beappreciated that software object 90B′ will be representative of thevideo image acquired by endoscope 90.

[0053] Furthermore, it will be appreciated that the two software objects90A′ and 90B′ will together represent both the physical structure ofendoscope 90 and the video image captured by that endoscope.

[0054] By way of example, in the case where digital computer 130comprises a Silicon Graphics computer of the sort described above, andwhere the operating system's software comprises the IRIX operatingsystem and the application program's image rendering software comprisesthe Iris gl or OpenGL image rendering software, the 3-D computer models190 might comprise software objects defined as polygonal surface models,since such a format is consistent with the aforementioned software.Furthermore, in a manner consistent with the aforementioned software, UVtexture mapping parameters are established for each of the vertices ofthe planar surface disk 90B′ and the digitized video signals fromendoscope 90 are assigned to be texture map images for 90B′. See, forexample, FIG. 10, which is a schematic representation of a unit disksoftware object where the disk is defined in the X-Y plane and has adiameter of 1.

[0055] It is important to recognize that, so long as the opticalcharacteristics of endoscope 90 remain constant, the size and positionalrelationships between shaft software object 90A′ and disk softwareobject 90B′ will also remain constant. As a result, it can sometimes beconvenient to think of shaft software object 90A′ and disk softwareobject 90B′ as behaving like a single unit, e.g., when positioning thesoftware objects 90A′ and 90B′ within 3-D computer models.

[0056] In accordance with the present invention, once the anatomical 3-Dcomputer models 160 have been established from anatomical softwareobjects 30A′, 30B′ and 30C′ (representative of the anatomical objects30A, 30B, and 30C which are to be visualized by the system), and oncethe endoscope 3-D computer models 190 have been established from theendoscope software objects 90A′ and 90B′ (representative of theendoscope and the video image captured by that endoscope), the varioussoftware objects are placed into proper registration with one anotherusing techniques well known in the art so as to form a cohesive databasefor the application program's image rendering software.

[0057] Stated another way, a principal task of the application programis to first resolve the relative coordinate system of all the varioussoftware objects of anatomical 3-D computer models 160 and of endoscope3-D computer models 190, and then to use the application program's imagerendering software to merge these elements into a single composite imagecombining both live video images derived from endoscope 90 with computergenerated images derived from the computer graphics system.

[0058] In this respect it will be appreciated that anatomical softwareobjects 30A′, 30B′ and 30C′ will be defined in 3-D computer models 160in the context of a particular coordinate system (e.g., the coordinatesystem established when the anatomical software objects were created),and endoscope software objects 90A′ and 90B will be defined in thecontext of the coordinate system established by endoscope tracking means50.

[0059] Various techniques are well known in the art for establishing theproper correspondence between two such coordinate systems. By way ofexample, where anatomical objects 30A′, 30B′ and 30C′ include uniquepoints of reference which are readily identifiable both visually andwithin the anatomical 3-D computer models 160, the tracked endoscope canbe used to physically touch those unique points of reference; suchphysical touching with the tracked endoscope will establish the locationof those unique points of reference within the coordinate system of theendoscope, and this information can then be used to map the relationshipbetween the endoscope's coordinate system and the coordinate system ofthe 3-D computer models 160. Alternatively, proper software objectregistration can also be accomplished by pointing endoscope 90 atvarious anatomical objects 30A, 30B and 30C and then having the systemexecute a search algorithm to identify the “virtual camera” positionthat matches the “real camera” position. Still other techniques forestablishing the proper correspondence between two such coordinatesystems are well known in the art.

[0060] Once the proper correspondence has been established between allof the coordinate systems, anatomical software objects 30A′, 30B′ and30C′ and endoscope software objects 90A′ and 90B′ can be considered tosimultaneously coexist in a single coordinate system in the mannerschematically illustrated in FIG. 4, whereby the application program'simage rendering software can generate images of all of the system'ssoftware objects (e.g., 30A′, 30B′, 30C′, 90A′ and 90B′) according tosome specified point of view.

[0061] Furthermore, inasmuch as the live video output from endoscope 90is texture mapped onto the surface of disk 90B′, the images generated bythe application program's image rendering software will automaticallyintegrate the relatively narrow field of view, live video image dataprovided by endoscope 90 with (ii) the wider field of view, computermodel image data which can be generated by the system's computergraphics. See, for example, FIG. 5, which shows a composite image 200which combines video image data 210 obtained from endoscope 90 withcomputer model image data 220 generated by the system's computergraphics.

[0062] It is to be appreciated that, inasmuch as endoscope trackingmeans 50 are adapted to continuously monitor the current position ofendoscope 90 and report the same to digital computer 130, digitalcomputer 130 can continuously update the 3-D computer models 190representing endoscope 90. As a result, the images generated by theapplication program's image rendering software will remain accurate evenas endoscope 90 is moved about relative to anatomical objects 30.

[0063] In addition to the foregoing, it should also be appreciated thatanatomical object tracking means 230 (comprising a tracking system 240generally similar to the tracking system 97 described above) may beattached to one or more of the anatomical objects 30A, 30B and 30C andthen used to advise computer 130 of the current position of thatanatomical object (see, for example, FIG. 1, where tracking system 240has been attached to the patient's tibia 30A and femur 30B). As aresult, digital computer 130 can continually update the 3-D computermodels 160 representing the anatomical objects. Accordingly, the imagesgenerated by the application program's image rendering software willremain accurate even as tibia 30A and/or femur 30B move about relativeto endoscope 90.

[0064]FIG. 6 provides additional details on how various elements of,system data are input into computer means 60 in connection with thesystem's generation of output video for display on video display 170.

[0065]FIG. 7 provides additional details on the methodology employed byanatomical visualization system 10 in connection with rendering a videooutput image for display on video display 170.

[0066] The application program software 140 of computer means 60 isconfigured so as to enable physician 20 to quickly and easily specify aparticular viewing position (i.e., a “virtual camera” position incomputer graphics terminology) for which the application program's imagerendering software should render a visualization of the 3-D softwaremodels contained in database means 70. By way of illustration, FIG. 8shows a typical Apple Newton screen display 300 which provides varioususer input choices for directing the system'“virtual camera”. Forexample, physician 20 may select one of the joystick modes as showngenerally at 310 for permitting the user to use a joystick-type inputdevice to specify a “virtual camera” position for the system.Alternatively, physician 20 may choose to use physician tracking means185 or 187 to specify the virtual camera position for the system, inwhich case movement of the physician will cause a corresponding changein the virtual camera position. Using such tools, the physician mayspecify a virtual camera position disposed at the very end of theendoscope's shaft, whereby the endoscope's shaft is not seen in therendered image (see, for example, FIG. 5), or the user may specify avirtual camera position disposed mid-way back along the length of theshaft, whereby a portion of the endoscope's shaft will appear in therendered image. See, for example, FIG. 9, which shows a composite image320 which combines video image data 330 obtained from endoscope 90 withcomputer model image data 340 generated by the system's computergraphics, and further wherein a computer graphic representation 350 ofthe endoscope's shaft appears on the rendered image.

[0067] It is to be appreciated that physician 20 may specify a virtualcamera position which is related to the spatial position and orientationof endoscope 90, in which case the virtual camera position will move inconjunction with endoscope 90. Alternatively, physician 20 may specify avirtual camera position which is not related to the spatial position andorientation of endoscope 90, in which case the virtual camera positionwill appear to move independently of endoscope 90.

[0068] Still referring now to FIG. 8, it is also possible for physician20 to use slider control 360 to direct the application program's imagerendering software to adjust the field of view set for the computergraphic image data 340 (see FIG. 9) generated by the system's computergraphics.

[0069] Additionally, it is also possible for physician 20 to use slidercontrol 370 to direct the application program's image rendering softwareto fade the density of the video image which is texture mapped onto theface of disk software object 90B′. As a result of such fading, the faceof the system's disk can be made to display an.overlaid composite madeup of both video image data and computer graphic image data, with therelative composition of the image being dictated according to the levelof fade selected.

[0070] It is also to be appreciated that, inasmuch as the display imagerendered by anatomical visualization system 10 is rendered from acollection of software objects contained in 3-D computer models, it ispossible to render the display image according to any preferred verticalaxis. Thus, for example, and referring now to control 380 in FIG. 8, itis possible to render the display image so that endoscope 90 providesthe relative definition of “up”, or so that the real world provides therelative definition of “up”, or so that some other object (e.g., theprincipal longitudinal axis of the patient's tibia) provides therelative definition of “up”.

[0071] It is also to be appreciated that anatomical visualization system10 can be configured to work with video acquisition devices other thanendoscopes. For example, the system can be configured to work withminiature ultrasound probes of the sort adapted for insertion into abody cavity. In this situation the video output of the miniatureultrasound probe would be texture mapped onto the face of disk softwareobject 90B′. Alternatively, other types of video acquisition devicescould be used in a corresponding manner.

[0072] Also, it is possible to use the foregoing visualization system torender images of objects other than anatomical structures. For example,the system would be used to provide images from the interior of complexmachines, so long as appropriate 3-D computer models are provided forthe physical structures which are to be visualized.

[0073] It is also to be understood that the present invention is by nomeans limited to the particular construction herein set forth, and/orshown in the drawings, but also comprises any modifications orequivalents within the scope of the claims.

What is claimed is:
 1. A real-time computer-based viewing systemcomprising: a database of pre-existing software objects, wherein atleast one of said software objects corresponds to a physical structurewhich is to be viewed by said system; a real-time sensor for acquiringdata about said physical structure when said physical structure islocated within that sensor's data acquisition field, wherein said sensoris capable of being moved about relative to said physical structure;generating means for generating a real-time software objectcorresponding to said physical structure using data acquired by saidsensor; registration means for positioning said real-time softwareobject in registration with said pre-existing software objects containedin said database; and processing means for generating an image from saidsoftware objects contained in said database, based upon a specifiedpoint of view.
 2. A system according to claim 1 wherein said generatingmeans creates a software object corresponding to a disk.
 3. A,systemaccording to claim 2 wherein said generating means are adapted totexture map the data acquired by said sensor onto said disk.
 4. A systemaccording to claim 1 wherein said registration means comprise trackingmeans for determining the spatial positioning and orientation of saidreal-time sensor.
 5. A system according to claim 1 wherein saidregistration means comprise tracking means for determining the spatialpositioning and orientation of said physical structure.
 6. A systemaccording to claim 1 wherein said system further comprises user inputmeans for permitting the user to provide said processing means with saidspecified point of view.
 7. A system according to claim 1 wherein saidsystem further comprises user tracking means for providing saidprocessing means with said specified point of view.
 8. A systemaccording to claim 1 wherein said real-time sensor comprises anendoscope.
 9. A system according to claim 1 wherein said physicalstructure comprises an interior anatomical structure.
 10. A real-timecomputer-based viewing system comprising: a database of softwareobjects; image generating means for generating an image from saidsoftware objects contained in said database, based upon a specifiedpoint of view; and means for specifying a point of view; wherein atleast one of said software objects contained in said database comprisespre-existing data corresponding to a physical structure which is to beviewed by said system; and wherein at least one of said software objectscomprises data generated by a real-time, movable sensor; and whereinsaid system further comprises registration means for positioning said atleast one software object comprising data generated by said real-time,movable sensor in registration with said at least one software objectcomprising pre-existing data corresponding to said physical structurewhich is to be viewed by said system.
 11. A system according to claim 10wherein said at least one software object comprising data generated bysaid real-time, movable sensor corresponds to a disk.
 12. A systemaccording to claim 11 wherein said data generated by said real-time,movable sensor is texture mapped onto said disk.
 13. A system accordingto claim 10 wherein said registration means comprise tracking means fordetermining the spatial positioning and orientation of said real-time,movable sensor.
 14. A system according to claim 10 wherein saidregistration means comprise tracking means for determining the spatialpositioning and orientation of said physical structure.
 15. A systemaccording to claim 10 wherein said means for specifying a point of viewcomprises user input means.
 16. A system according to claim 10 whereinsaid means for specifying a point of view comprises user tracking means.17. A system according to claim 10 wherein said real-time, movablesensor comprises an endoscope.
 18. A system according to claim 10wherein said physicial structure comprises an interior anatomicalstructure.
 19. A method for viewing a physical structure, said methodcomprising the steps of: (A) providing: a database of pre-existingsoftware objects, wherein at least one of said software objectscorresponds to a physical structure which is to be viewed by saidsystem; a real-time sensor for acquiring data about said physicalstructure when said physical structure is located within that sensor'sdata acquisition field, wherein said sensor is capable of being movedabout relative to said physical structure; generating means forgenerating a real-time software object corresponding to said physicalstructure using data acquired by said sensor; registration means forpositioning said realtime software object in registration with saidpreexisting software objects contained in said database; and; processingmeans for generating an image from said software objects contained insaid database, based upon a specified point of view; (B) positioningsaid sensor so that said physical structure is located within thatsensor's data acquisition field, and generating a real-time softwareobject corresponding to said physical structure using data acquired bysaid sensor, and positioning said real-time software object inregistration with said pre-existing software objects contained in saiddatabase; (C) providing a specified point of view to said processingmean; and (D) generating an image from said software objects containedin said database according to said specified point of view. 20.Apparatus according to claim 3 wherein said generating means comprisesmeans for varying the relative density of the data which is texturemapped onto said disk, whereby the portion of the image generated bysaid processing means which is attributable to data acquired by saidreal-time sensor can be faded relative to the remainder of the imagegenerated by said processing means.
 21. A system according to claim 1wherein said generating means creates a software object corresponding toa disk.
 22. A system according to claim 21 wherein said generating meansare adapted to texture map the data acquired by said sensor onto saiddisk.
 23. A system according to claim 10 wherein said at least onesoftware object comprising data generated by said real-time, movablesensor corresponds to a disk.
 24. A system according to claim 23 whereinsaid data generated by said real-time, movable sensor is texture mappedonto said disk.